Nonadditive Conditional Entropy and 
Its Significance for Local Realism 



Sumiyoshi Abe (1) and A. K. Rajagopal (2) 

{\)College of Science and Technology, Nihon University, 
Funabashi, Chiba 274-8501, Japan 
{2)N aval Research Laboratory, Washington, D. C. 20375-5320, USA 



Based on the form invariance of the structures given by Khinchin's axiomatic 
foundations of information theory and the pseudoadditivity of the Tsallis entropy 
indexed by q, the concept of conditional entropy is generalized to the case of 
nonadditive (nonextensive) composite systems. The proposed nonadditive conditional 
entropy is classically nonnegative but can be negative in the quantum context, 
indicating its utility for characterizing quantum entanglement. A criterion deduced from 
it for separability of density matrices for validity of local realism is examined in detail 
by employing a bipartite spin- 1/2 system. It is found that the strongest criterion is 
obtained in the limit ^ — > oo . 
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In this Letter, we present a generalization of the concept of "conditional entropy" for 
nonadditive (nonextensive) composite systems. This is done based on the form 
invariance of the structures in Khinchin's axiomatic foundations of classical 
information theory and the pseudoadditivity of the Tsallis entropy indexed by q in. 
nonextensive statistical mechanics. Then, we discuss the nonadditive conditional 
entropy in the quantum context. It is shown that the nonadditive conditional entropy is 
nonnegative for all classically correlated states (i.e., separable states) but can be 
negative for nonclassical correlated states (i.e., quantum entangled states). Since 
classically correlated states admit local hidden-variable models, the positivity of the 
nonadditive conditional entropy leads to a criterion for validity of local realism. This 
criterion depends on the Tsallis nonadditivity parameter q. We shall see how the 
strongest criterion can be obtained by controlling the value of q. The present attempt 
can also be thought of as a stepping stone towards nonadditive (quantum) information 
theory. 

In the development of the foundations of classical information theory, Khinchin [1] 

presented a mathematically rigorous proof of a uniqueness theorem for the Boltzmann- 

Shannon entropy based on the additivity law for a composite system in terms of the 

concept of conditional entropy. Suppose the total system be divided into two 
subsystems, A and B, and let p,^ (A, E) be the normalized joint probability of finding 

A and B in their ith and jth states, respectively. Then the conditional probability of B 
with A found in its fth state is given by Pj^{B\A^= Pij{A, B)l p,(A), which leads to the 

celebrated Bayes multiplication law 

p^.{A,B) = p,{A)p,.(B\A\ (1) 
where /7, (A) is the marginal probability distribution: p,(A) = ^ . /7,y(A, B). It should be 

noted that this form of factorization can always be established in any physical situation. 
From this law, the Boltzmann-Shannon entropy 

S{A, B) = PiM' B) lnp,,(A, B) (2) 



S{A,B) = S{A) + S{b\a), (3) 
where 5(5 |A) stands for the conditional entropy defined by 

S(B\ A) = {s{b\ a)/'' = pM)S(b\ a) (4) 

i 

with 

S{b\A)^-Y, Pu(B\A)lnp,.(B\A). (5) 

j 

In the particular case when A and B are statistically independent, Pjj[B\A) is equal to 

Pj{B) and therefore S{b\a)=S{B), implying the additivity law: .S{A,B) = S{A) 

+ S{B). We emphasise here that there is a natural correspondence relation between the 

multiplication law and the additivity law: 

P,j{A,B) = p,{A)p,j{b\a) ^ S{A,B) = S{A) + S{b\a). (6) 

When the above discussion is generalized to quantum theory of composite systems, a 
remarkable feature appears. The conditional entropy may then take negative values, 
suggesting its importance for characterizing quantum entanglement [2]. This feature 
may have its root in a profound difference between classical and quantum probability 
concepts: in the latter, one of Kolmogorov's axioms, namely the additivity of the 
probability measure, is violated in general [3]. Also, there are theoretical observations 
[4,5] that formal correspondences exist between thermodynamics and quantum 
entanglement. These investigations seem to suggest that the measure of quantum 
entanglement may not be additive [5,6]. 

Nonadditivity, or nonextensivity, is an important concept also in the field of 
statistical mechanics. A statistical system is nonextensive if it contains long-range 
interaction, long-range memory, or (multi)fractal structure. In such a system, a 
macroscopic thermodynamic quantity (e.g., the internal energy) is not simply 
proportional to the microscopic degrees of freedom. Boltzmann-Gibbs statistical 
mechanics, which can be constructed based on the additive Boltzmann-Shannon 
entropy is known to expose difficulties, if applied to such a system [7]. In this respect, a 
nonextensive generalization of Boltzmann-Gibbs statistical mechanics formulated by 



Tsallis [8,9] has attracted much attention in recent years. In this formalism, referred to 
as nonextensive statistical mechanics, the Boltzmann-Shannon entropy in eq. (2) is 
generalized as follows: 



where ^ is a positive parameter. This quantity converges to the Boltzmann-Shannon 

entropy in the limit ^— >1. Like the Boltzmann-Shannon entropy, it is nonnegative, 

possesses the definite concavity for all ^ > 0, and is known to satisfy the generalized H- 

theorem. Nonextensive statistical mechanics has found a lot of physical applications. (A 

comprehensive list of references is currently available at URL [10]. See also Ref. [11].) 
A standard discussion about the nonadditivity of the Tsallis entropy S^[p] assumes 

factorization of the joint probability distribution in eq. (2). Then, the Tsallis entropy is 
found to yield the so-called pseudoadditivity relation 



Clearly, the additivity holds if and only if ^ — > 1. However, there is a logical difficulty 
in this discussion. As mentioned above, Tsallis' nonextensive statistical mechanics was 
devised in order to treat a statistical system with, for example, a long-range interaction. 
On the other hand, physically, "dividing the total system into the subsystems" implies 
that the subsystems are made spatially separated in such a way that there is no residual 
interaction or correlation. If the system is governed by a long-range interaction, the 
statistical independence can never be realized by any spatial separation since the 
influence of the interaction persists at all distances. In fact, the probability distribution 
in nonextensive statistical mechanics does not have a factorizable form even if the 
systems A and B are dynamically independent, and therefore correlation is always 
induced by nonadditivity of statistics [12]. Another important physical mechanism 
which prevents realizing the statistical independence by spatial separation is provided 
by the concept of entanglement in quantum theory. Thus, it is clear that the assumption 
of the factorized joint probability distribution is not physically pertinent for 




(7) 



S^{A, B) = S^{A) + S^{B) + (1 - q)S^{A)S^{B). 



(8) 



characterizing the nonadditivity of the Tsallis entropy. These considerations naturally 
lead us to the necessity of defining the conditional entropy associated with the Tsallis 
entropy. 

To overcome the above-mentioned logical difficulty and to generalize the 
correspondence relation in eq. (6) simultaneously, first let us recall the definition of the 
normalized ^-expectation value in nonextensive statistical mechanics [9]. In order to 
specify the constraint on a physical quantity to develop the Jaynes maximum entropy 
principle for nonextensive statistical mechanics based on the Tsallis entropy, the 
authors of Ref. [9] introduced the idea of the normalized ^-expectation value. Consider 
a physical quantity Q, whose value in the system's /th state is denoted by 2,. Its 

normalized ^-expectation value is defined by 

<<2>,=L^a, (9) 

i 

where P^ is the so-called escort distribution [13] associated with the original probability 
distribution p^^. 



(10) 



i 

Next, let us consider the Tsallis entropy of the conditional probability distribution 
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(a (A))' 



-1 



(11) 



which is a natural nonadditive generalization of eq. (5). Now, in conformity with eq. 

(4), we propose to define the nonadditive conditional entropy as the normalized q- 
expectation value of 5^(5|A,) with respect to the marginal probability distribution 

Pi{A) as follows: 



£(p,(A)rs,(s|A,) 

- ' I (P^r ■ 

Using eq. (11) in eq. (12), we obtain 

, , , SiA,B)-SiA) 
sXb\A)= \ ' (13) 

This is our definition of the nonadditive conditional entropy. From this, it is immediate 
to see 

S^A, B) = S^{A) + Sp\ A) + (1 - q)S^{A)Sp\ A) , (14) 

which is a natural nonadditive generalization of eq. (3) in view of the pseudoadditivity 
in eq. (8). Therefore, the correspondence relation in eq. (6) is generalized to 

77,,(A,5) = p,(A)p,,(5|A) ^ 

S^{A, B) = S^{A) + Sp\ A) + (1 - q) S^{A) Sp\ A) . (15) 
In quantum theory, the probability distribution is replaced by the density matrix p, 
which is Hermitian, traceclass, and positive semidefinite. It incorporates both pure and 
mixed states of a system. When the state is pure, the equality = p holds, whereas 
p^ < p for the mixed state. The quantum counterpart of eq. (7) is given by 

S^{A, B) = -i-[Tr(p(A, B))' - 1], (16) 

which converges to the von Neumann entropy S{A, B) = -Tr[p(A, B) In p(A, 5)] in the 
limit q^l. It is evident that, like the von Neumann entropy, this quantity is 
nonnegative and vanishes for a pure state. 

There is an ordering ambiguity in defining the "conditional density matrix", since the 
joint density matrix and its marginals do not commute with each other, in general. Here, 
we avoid to define it explicitly and directly translate eq. (13) to its quantum counterpart. 
That is, 

, , , S(A,B)-S(A) 



with 

^ ' '- - ' (18) 



l-qi- 



and Sg{A, B) given in eq. (16). In eq. (18), p(A) is the marginal density operator 
defined by p(A) = Tr^ p(A, B), where Tr^ (Tr^) stands for the partial trace over the 
states of the subsystem A ( 5). By going to the representations in which p(A, B) and 
p(A) are diagonal, S^{b\A) appears to take the form of eq. (12). The difference 

between the two is now that the expansion coefficients are the eigenvalues of the 

diagonalized density matrices in contrast to their classical counterparts presented 
earlier. In the limit ^ — > 1, S^{b\A) converges to the conditional von Neumann entropy 

S(B I a) = 5(A, B) - S{A) discussed in detail in Ref. [2]. 

Let us consider two particular cases: a product state and a classically correlated state. 
A product state p(A, B) = p(A) ® p(B), gives rise to the pseudoadditivity relation, 
whose form is the same as the classical one in eq. (8). Therefore, we have S^(5|a) = 
S^{B) as expected. A classically correlated state, i.e. a separable state, is a convex 

combination of product states [14]: 

p(A,fi) = J^w,p,(A)®p,(5), (19) 

where e [0, 1] and ^1=^- This state is known to admit local hidden-variable 

models and to satisfy the Bell inequality [14]. Using the orthonormal bases of A and5, 
we write 

i^A(^) = L Px(^)\^)H UB) = L 'x(b)\b){b\, (20) 
where p^(a), r^(b)s [0, 1] and Px((i) = = 1. In this case, the nonadditive 

quantum conditional entropy is given by 

a 

provided that we have used the notation '^^(^l'^) = (1~ ?)^|]^^ "^^^^ 
7r[b I a) = p^(a)r^(b) Pi(a)- Note that 7r[b\a) e [0, 1] and 7r[b \a) = l. 



Therefore 7t[b\a) is analogous to the classical conditional probability distribution. This 
fact makes 5^(5 |a) nonnegative. Thus, we conclude that 5^(5|A)>0 for any 

classically correlated states. This establishes a ^-generalization of the result in Ref. [2]. 
Here, it is worth mentioning that in Ref. [15] the Renyi entropy 5^[p] = (1- a)"' 

xlnTr(p)" with a>l is discussed in connection with violation of local realism in 

quantum theory. The authors of Ref. [15] proposed the a-entropic inequality: 
S^(A, B) > max{5f (A), S^{B)\, which can be violated by entangled mixed states. We 

note that the Renyi entropy is additive but is not concave for a > 1 . 

A corresponding entropic inequality in the present nonnadditive conditional entropy 

is 

5^(5|A)>0, SXA\B)>Q. (22) 

As mentioned above, eq. (22) holds for all classically correlated states that can be 

modelled by local hidden-variable theories. Nonclassical correlated states, that is, 
quantum entangled states, yield S^{b\A)<Q. Therefore, the inequality (22) can be used 

as a criterion for separability of density matrices. A point is that the inequality depends 
on the parameter q . 

To see how the separability criterion is strengthened by controlling the value of q , 
let us employ a bipartite spin- 1/2 system. For this purpose, we consider the following 
parametrized form of the Wemer-Popescu state [14,15]: 

p(A, B) = ^l<S>I, + x |^-){^- 1, (23) 
where < x < 1, 1/^,) the 2x2 identity matrix in the space of the spin A (B), and 




The separability criteria known so far are: a) x<l/-\j2 (the Bell inequality), b) 
X < 1/ V3 (the a-entropic inequality with a = 2) [16], and c) x < 1/3 (the nonnegative 
eigenvalues of the density matrix with partial transpose) [17]. The strongest criterion c) 
found by Peres [17] is known to actually be the necessary and sufficient condition for 

thf spnarahilitv nf thp hinartitp sr>in-1 /9 svstptn Tl Rl Our intprpst hprp is tr> spp whpthpr 



the Peres criterion obtained by an algebraic method can be derived within the present 
framework of nonadditive quantum information theory. From the density matrix in eq. 
(23), the nonadditive conditional entropy is calculated to be 

For a fixed value of q, this is a monotonically decreasing function of x. In Fig. 1, we 
present an implicit plot of 5^(5 |A) = with respect to ^ € [0, oo) and x e [0, 1]. One 

clearly sees how the criterion obtained from the conditional von Neumann entropy 
(^— >1) can be strengthened by increasing the value of q. From eq. (25), it is also 
evident that S^[b\A) ^ in the limit ^ — > oo if and only if x < 1/3. Thus, the Peres 

strongest criterion is obtained from the present information-theoretic approach. 

In conclusion, we have constructed the nonadditive conditional entropy based on the 
form invariance of the structures of the axiomatic foundations of classical information 
theory and the pseudoadditivity of the Tsallis entropy indexed by q. Then, we have 
discussed it in the quantum context and applied it to separability of the density matrix 
for validity of local realism. We have found that for a bipartite spin- 1/2 system, i.e., a 
2x2 system, the positivity of the nonadditive conditional entropy leads to the Peres 
strongest criterion for separability in the limit ^— >oo. In Ref. [18], it has been 
discussed that Peres' s method of partial transposition of the density matrix yields the 
necessary and sufficient condition for separability of 2x2 and 2x3 systems but not 
for other general systems. Therefore, for a further development, it is important to clarify 
the general properties of the condition in eq. (22). 
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Figure Caption 



An implicit plot of 5^(5 |a) = with respect to ^ e [0, oo) and x e [0, 1]. In the limit 
^ — > oo, X converges to 1/3. 
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